{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Private Predictions with TFE Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 1: Public Training without Differential Privacy\n",
    "\n",
    "Welcome to this tutorial! In the following notebooks you will learn how to provide private predictions. By private predictions, we mean that the data is constantly encrypted throughout the entire process. At no point is the user sharing raw data, only encrypted (that is, secret shared) data. In order to provide these private predictions, [TF Encrypted](https://github.com/tf-encrypted/tf-encrypted) combines cutting-edge cryptographic and machine learning techniques. But you don't have to worry about this and can focus on your machine learning application.\n",
    "\n",
    "You can start serving private predictions with only three steps:\n",
    "- **Step 1**: train your model with normal Keras.\n",
    "- **Step 2**: secure and serve your machine learning model (server).\n",
    "- **Step 3**: query the secured model to receive private predictions (client). \n",
    "\n",
    "Alright, let's go through these three steps so you can deploy impactful machine learning services without sacrificing user privacy or model security."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train your model with Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use privacy-preserving machine learning techniques for your projects you should not have to learn a new machine learning framework. If you have basic [Keras](https://keras.io/) knowledge, you can start using these techniques with Syft Keras. If you have never used Keras before, you can learn a bit more about it through the [Keras documentation](https://keras.io). \n",
    "\n",
    "Before serving private predictions, the first step is to train your model with normal Keras. As an example, we will train a model to classify handwritten digits. To train this model we will use the canonical [MNIST dataset](http://yann.lecun.com/exdb/mnist/).\n",
    "\n",
    "We borrow [this example](https://github.com/keras-team/keras/blob/master/examples/mnist_cnn.py) from the reference Keras repository.  To train your classification model, you just run the cell below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Public Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda3/envs/py35_int64/lib/python3.5/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "import tensorflow as tf\n",
    "\n",
    "from tensorflow.keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28, 1)\n",
      "60000 train samples\n",
      "10000 test samples\n",
      "WARNING:tensorflow:From /anaconda3/envs/py35_int64/lib/python3.5/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n",
      "Train on 60000 samples, validate on 10000 samples\n",
      "WARNING:tensorflow:From /anaconda3/envs/py35_int64/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n",
      "Epoch 1/2\n",
      "60000/60000 [==============================] - 5s 83us/sample - loss: 0.4994 - acc: 0.8462 - val_loss: 0.1667 - val_acc: 0.9534\n",
      "Epoch 2/2\n",
      "60000/60000 [==============================] - 5s 78us/sample - loss: 0.1561 - acc: 0.9545 - val_loss: 0.1133 - val_acc: 0.9679\n",
      "Test loss: 0.11331137989088894\n",
      "Test accuracy: 0.9679\n"
     ]
    }
   ],
   "source": [
    "batch_size = 128\n",
    "num_classes = 10\n",
    "epochs = 2\n",
    "\n",
    "# input image dimensions\n",
    "img_rows, img_cols = 28, 28\n",
    "\n",
    "# the data, split between train and test sets\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
    "\n",
    "x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)\n",
    "x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)\n",
    "input_shape = (img_rows, img_cols, 1)\n",
    "\n",
    "x_train = x_train.astype('float32')\n",
    "x_test = x_test.astype('float32')\n",
    "x_train /= 255\n",
    "x_test /= 255\n",
    "print('x_train shape:', x_train.shape)\n",
    "print(x_train.shape[0], 'train samples')\n",
    "print(x_test.shape[0], 'test samples')\n",
    "\n",
    "# convert class vectors to binary class matrices\n",
    "y_train = tf.keras.utils.to_categorical(y_train, num_classes)\n",
    "y_test = tf.keras.utils.to_categorical(y_test, num_classes)\n",
    "\n",
    "model = tf.keras.Sequential([\n",
    "          tf.keras.layers.Conv2D(16, 8,\n",
    "                                 strides=2,\n",
    "                                 padding='same',\n",
    "                                 activation='relu',\n",
    "                                 input_shape=(28, 28, 1)),\n",
    "          tf.keras.layers.AveragePooling2D(2, 1),\n",
    "          tf.keras.layers.Conv2D(32, 4,\n",
    "                                 strides=2,\n",
    "                                 padding='valid',\n",
    "                                 activation='relu'),\n",
    "          tf.keras.layers.AveragePooling2D(2, 1),\n",
    "          tf.keras.layers.Flatten(),\n",
    "          tf.keras.layers.Dense(32, activation='relu'),\n",
    "          tf.keras.layers.Dense(10, activation='softmax')\n",
    "  ])\n",
    "\n",
    "model.compile(loss=tf.keras.losses.categorical_crossentropy,\n",
    "              optimizer=tf.keras.optimizers.Adam(),\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "model.fit(x_train, y_train,\n",
    "          batch_size=batch_size,\n",
    "          epochs=epochs,\n",
    "          verbose=1,\n",
    "          validation_data=(x_test, y_test))\n",
    "score = model.evaluate(x_test, y_test, verbose=0)\n",
    "print('Test loss:', score[0])\n",
    "print('Test accuracy:', score[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save your model's weights for future private prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Your model is trained. Let's save the model weights with `model.save()`. In the next notebook, we will load these weights in Syft Keras to start serving private predictions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save('short-dnn.h5')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
